Information processing device, information processing system, and information processing method

ABSTRACT

Disclosed herein is an information processing device including an acquiring unit that acquires positional information of a flat surface present in a first space around a first user and positional information of a flat surface present in a second space around a second user, and a transformation parameter determining unit that determines a coordinate transformation parameter for transforming position coordinates of the first space and the second space into position coordinates in a virtual space such that a position of the flat surface present in the first space and a position of the flat surface present in the second space coincide with each other. A position of an object present in the first space and a position of another object present in the second space are transformed into positions in the virtual space according to the determined coordinate transformation parameter.

CROSS REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of Japanese Priority PatentApplication JP 2019-170720 filed Sep. 19, 2019, the entire contents ofwhich are incorporated herein by reference.

BACKGROUND

The present disclosure relates to an information processing device, aninformation processing system, and an information processing method forperforming processing relating to a virtual space.

In recent years, in a technology such as virtual reality or augmentedreality, it has been studied to reflect, when a user moves his or herbody in an actual world, the movement on, for example, an avatar in avirtual space. According to such a technology, the user moves his or herbody in the actual world while viewing a state in the virtual space witha display device such as a head mounted display being mounted. Thisallows the user to, for example, touch an object present in the virtualspace and to enjoy experiences in the virtual space with high presence.

SUMMARY

In the above-described technology, a plurality of users present inplaces spaced away from each other shares an identical virtual space insome cases. How to appropriately build the virtual space in such caseshas not ever been studied sufficiently.

The present disclosure has been made in consideration of the abovesituation, and it is desirable to provide an information processingdevice, an information processing system, and an information processingmethod capable of suitably building a virtual space to be shared by aplurality of users present in different places.

According to an embodiment of the present disclosure, there is providedan information processing device including an acquiring unit thatacquires positional information of a flat surface present in a firstspace around a first user and positional information of a flat surfacepresent in a second space around a second user, and a transformationparameter determining unit that determines a coordinate transformationparameter for transforming position coordinates of the first space andthe second space into position coordinates in a virtual space such thata position of the flat surface present in the first space and a positionof the flat surface present in the second space coincide with eachother. A position of an object present in the first space and a positionof another object present in the second space are transformed intopositions in the virtual space according to the determined coordinatetransformation parameter.

According to another embodiment of the present disclosure, there isprovided an information processing system including a plurality ofclient devices and a server device. The server device includes a flatsurface position acquiring unit that acquires, from each of theplurality of client devices, positional information of a flat surfacepresent in a space around a user of the client device, and atransformation parameter determining unit that determines a coordinatetransformation parameter for transforming a position coordinate of aspace around a user who uses corresponding one of the plurality ofclient devices into a position coordinate in a virtual space such thatpositions of the flat surfaces acquired from the plurality of clientdevices coincide with each other. Each of the plurality of clientdevices includes an object position acquiring unit that acquires aposition coordinate of an object present in a space around a user whouses the client device, and a coordinate transformation unit thattransforms the position coordinate of the object into a position in thevirtual space according to the coordinate transformation parameter.

According to a further embodiment of the present disclosure, there isprovided an information processing method including acquiring positionalinformation of a flat surface present in a first space around a firstuser and positional information of a flat surface present in a secondspace around a second user, and determining a coordinate transformationparameter for transforming position coordinates of the first space andthe second space into position coordinates in a virtual space such thata position of the flat surface present in the first space and a positionof the flat surface present in the second space coincide with eachother. A position of an object present in the first space and a positionof another object present in the second space are transformed intopositions in the virtual space according to the determined coordinatetransformation parameter.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic view of an entire information processing systemincluding an information processing device according to an exemplaryembodiment of the present disclosure;

FIGS. 2A and 2B are views illustrating examples of a state of an actualspace where a user is present;

FIG. 3 is a functional block diagram illustrating functions of theinformation processing system;

FIG. 4 is a view illustrating an example of a state of a virtual space;and

FIG. 5 is a flow diagram illustrating an example of a flow of processingperformed by the information processing system.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

Hereinafter, an exemplary embodiment of the present disclosure will bedescribed in detail with reference to the drawings.

FIG. 1 is a schematic view of an entire information processing system 1including a server device 10 that is an information processing deviceaccording to the exemplary embodiment of the present disclosure. Theinformation processing system 1 includes the server device 10 and aplurality of client devices 20 as illustrated in FIG. 1. Each of theplurality of client devices 20 is connected to the server device 10through a communication network such as the Internet in adata-communicatable manner. Herein, as a specific example, it is assumedthat the information processing system 1 includes two client devices 20that are a client device 20 a used by a user A and a client device 20 bused by a user B. However, the information processing system 1 is notlimited thereto and may include three or more client devices 20.

The server device 10 is an information processing device including acontroller 11, a storage 12, and a communication unit 13. The controller11 includes at least one processor and executes a program stored in thestorage 12 to perform various kinds of information processing. Aspecific example of the processing performed by the controller 11 in thepresent exemplary embodiment will be described later. The storage 12includes at least one memory device such as a random access memory (RAM)and stores the program to be executed by the controller 11 and data tobe processed by the program. The communication unit 13 is acommunication interface for performing data communication with each ofthe client devices 20 through the communication network.

Each of the client devices 20 includes a controller 21, a storage 22, acommunication unit 23, and an interface 24. Further, each of the clientdevices 20 is connected with a display device 25 and a stereo camera 26.The controller 21 includes at least one processor and executes a programstored in the storage 22 to perform various kinds of informationprocessing. A specific example of the processing performed by thecontroller 21 in the present exemplary embodiment will be describedlater. The storage 22 includes at least one memory device such as a RAMand stores the program to be executed by the controller 21 and data tobe processed by the program. The communication unit 23 is acommunication interface for performing data communication with theserver device 10 through the communication network.

The interface 24 is an interface for performing data communication withthe display device 25 and the stereo camera 26. The client device 20 iswiredly or wirelessly connected to the display device 25 and the stereocamera 26 through the interface 24. Specifically, the interface 24includes a multimedia interface such as high-definition multimediainterface (HDMI) (registered trademark) to transmit data of movingimages depicted by the client device 20 to the display device 25.Further, the interface 24 includes a data communication interface suchas a universal serial bus (USB) to receive a video signal transmittedfrom the stereo camera 26.

The display device 25 is a device that displays a moving image to bepresented to the user. In the present exemplary embodiment, the displaydevice 25 is assumed to be a head mounted display device to be mountedon the user's head to use. Specifically, the display device 25 candisplay a three-dimensional video using parallax while presenting aright-eye image to the right eye of the user and presenting a left-eyeimage to the left eye of the user.

The stereo camera 26 includes a plurality of imaging elements disposedin an array. The client device 20 can measure a distance from an imagingposition to an object present in a viewing-field range of the stereocamera 26 by analyzing an image captured by each of these imagingelements. This allows the client device 20 to estimate a position or ashape of an object present in a space around the stereo camera 26.

Particularly, in the present exemplary embodiment, the stereo camera 26is used for identifying positions of a user who uses the client device20 and an object present around the user, such as a wall or a floor.FIGS. 2A and 2B are views each illustrating a state in which each useruses the client device 20. FIG. 2A illustrates a state of the user A,and FIG. 2B illustrates a state of the user B. In the examples in FIGS.2A and 2B, the stereo camera 26 is disposed in front of the user A withthe display device 25 mounted, and a viewing-field range of the stereocamera 26 includes the user A standing in front, a floor surface onwhich the user A stands, and a wall surface of a room where the user Ais present, for example. Similarly to the user A, the user B also usesthe client device 20 b in his or her room, and a viewing-field range ofthe stereo camera 26 connected to the client device 20 b includes theuser B, and a floor surface and a wall surface around the user B.

Functions to be implemented by the information processing system 1 willbe described with reference to FIG. 3. In the present exemplaryembodiment, the information processing system 1 provides a serviceallowing a plurality of users to mutually interact in a virtual space.As illustrated in FIG. 3, the information processing system 1functionally includes a positional information acquiring unit 51, areference surface identification unit 52, a space coupling unit 53, acoordinate transforming unit 54, a space updating unit 55, and a spatialimage depicting unit 56. Among these functions, the positionalinformation acquiring unit 51, the reference surface identification unit52, the coordinate transforming unit 54, and the spatial image depictingunit 56 are implemented by the controller 21 of each client device 20executing the program stored in the storage 22. The space coupling unit53 and the space updating unit 55 are implemented by the controller 11of the server device 10 executing the program stored in the storage 12.These programs may be provided to the respective devices through thecommunication network such as the Internet or may be provided by beingstored in a computer-readable non-transitory recording medium such as anoptical disk.

The positional information acquiring unit 51 acquires information thatidentifies positions of the user and an object present around the userin the actual space. In the present exemplary embodiment, the positionalinformation acquiring unit 51 analyzes images captured by the stereocamera 26 to identify a position of an object present in the imagingrange of the stereo camera 26. The positional information acquired bythe positional information acquiring unit 51 is used to identify areference surface by the reference surface identification unit 52 to bedescribed later and is also used to update a virtual space by the spaceupdating unit 55 to be described later.

More specifically, with respect to each of a plurality of unit regionsobtained by dividing the imaging range of the stereo camera 26 in a gridpattern, the positional information acquiring unit 51 calculates adistance to a portion of an object captured in each unit region. Thisallows the positional information acquiring unit 51 to acquire adistance image (depth map) including information of a distance to anobject captured in each unit region. Hereinafter, a portion, of theobject present in the actual space, captured in each unit region in thedistance image is referred to as a unit portion.

Furthermore, the positional information acquiring unit 51 calculatesposition coordinates of each unit portion in the actual space usingnumerical values of a position of the unit region in the distance image(i.e., a direction of the unit region viewed from an installationposition of the stereo camera 26) and a distance to a unit portioncaptured in the unit region. These position coordinates serve as athree-dimensional position coordinates in a coordinate system thatdefines the installation position of the stereo camera 26 as a referenceposition. By using the position coordinates of each of the plurality ofunit portions thus acquired, a position or a shape of an object having acomplicated structure, such as the body of the user or a table, can beidentified. Hereinafter, a coordinate system indicating positions in theactual space where the user is present with the position of the stereocamera 26 as the reference position is referred to as a local coordinatesystem. Further, a coordinate system indicating positions in an actualspace around the user A (first space) is referred to as a first localcoordinate system, and a coordinate system indicating positions in anactual space around the user B (second space) is referred to as a secondlocal coordinate system.

The reference surface identification unit 52 identifies a position ofthe reference surface in the actual space by use of the positionalinformation of the object which is acquired by the positionalinformation acquiring unit 51. Herein, the reference surface is a flatsurface serving as a reference when spaces are coupled, which will bedescribed later. The reference surface identification unit 52 extracts aregion where a plurality of unit portions are continuously disposedalong an identical flat surface by use of the position coordinates ofeach unit portion which are acquired by the positional informationacquiring unit 51, thereby to identify the flat surface configured withsuch unit portions as the reference surface.

Specifically, the reference surface identification unit 52 identifiesone horizontal surface as the reference surface. For example, thereference surface may be a floor surface on which the user stands.Alternatively, in a case where an object whose upper surface is flatsuch as a table is present in a location where the user is present, theupper surface may be identified as the reference surface.

Note that it is assumed that each of the plurality of client devices 20identifies a surface whose relative position (height) to the user isclose as the reference surface. For example, in a case where the clientdevice 20 a defines a floor surface in the location where the user A ispresent as the reference surface, the client device 20 b also identifiesa floor surface in the location where the user B is present as thereference surface. In contrast, in a case where the plurality of usersshares a table in the virtual space, for example, the reference surfaceidentification unit 52 of each client device 20 identifies an uppersurface of an actual table present close to the user as the referencesurface.

To achieve such control, the reference surface identification unit 52 ofeach client device 20 may accept, in advance, specification of acondition regarding what height of the horizontal surface should be usedas the reference surface from the server device 10 and may identify aflat surface that satisfies the condition as the reference surface. Forexample, in a case where the reference surface identification unit 52accepts specification for defining the floor surface as the referencesurface, the reference surface identification unit 52 defines a flatsurface configured with a unit portion present at a lowest position asthe reference surface. In a case where the reference surfaceidentification unit 52 accepts specification for defining a tablesurface as the reference surface, the reference surface identificationunit 52 may first identify a position of the head of the user and aposition of the floor surface from a distribution of unit portions andthen define a flat surface present in a predetermined height rangebetween the head of the user and the floor surface as the referencesurface.

Furthermore, the reference surface identification unit 52 may identify avertical surface such as a wall surface as the reference surface inaddition to the horizontal surface such as the floor surface. Herein, itis assumed that each user uses the client device 20 in the room, and thereference surface identification unit 52 identifies, as the referencesurface, at least one of some of four wall surfaces in the room wherethe user is present. Note that, hereinafter, the horizontal surfaceidentified as the reference surface is referred to as a horizontalreference surface, and the vertical surface identified as the referencesurface is referred to as a vertical reference surface. It is assumedthat the reference surface identification unit 52 typically identifiesone horizontal reference surface and further identifies one or morevertical reference surfaces if possible. For example, in the examplesillustrated in FIGS. 2A and 2B, the reference surface identificationunit 52 of the client device 20 a identifies a floor surface P1 a and awall surface P2 a as the reference surfaces. Further, the referencesurface identification unit 52 of the client device 20 b identifies afloor surface P1 b and a wall surface P2 b as the reference surfaces.The vertical reference surface may be a reference surface adjacent tothe horizontal reference surface or may be a reference surface presentat a position closest to the position of the user.

After identifying the position of the reference surface in the actualspace, the reference surface identification unit 52 transmits thepositional information to the server device 10. This positionalinformation is represented by coordinate values indicating a position ofa flat surface in the local coordinate system.

The space coupling unit 53, as a flat-surface position acquiring unit,builds the virtual space by use of information of the reference surfacestransmitted from each client device 20. More specifically, the spacecoupling unit 53 determines coordinate transformation parameters forcoupling the actual spaces where the users of the client devices 20 areindividually present to build the virtual space. The coordinatetransformation parameters are parameters for transforming the localcoordinate system used for representing the positional information ofthe object acquired by each client device 20 into a coordinate systemindicating a position in the virtual space (herein, referred to as aworld coordinate system). The positional information of the objectacquired by each client device 20 is transformed into the position inthe world coordinate system by use of the coordinate transformationparameters, which enables the plurality of users to share one virtualspace.

Herein, the space coupling unit 53 configures a transformation parameterdetermining unit that determines the coordinate transformationparameters such that at least the horizontal reference surfaces in thespaces where the plurality of users are individually present coincidewith each other. The position coordinates of the horizontal referencesurfaces in the local coordinate systems are different from each otherdue to, for example, a difference between the installation positions ofthe stereo cameras 26 connected to the respective client devices 20. Thespace coupling unit 53 determines the coordinate transformationparameters to be transmitted to the respective client devices 20 so asto cancel this difference. The coordinate transformation parameters inthis case at least include a parameter for a parallel motion along avertical direction for causing the horizontal reference surfaces tocoincide with each other.

Furthermore, in a case where each of the client devices 20 transmits thepositional information of the vertical reference surface, the spacecoupling unit 53 may couple the spaces such that the vertical referencesurfaces at least partly coincide with each other. As an example, in acase where each of the client devices 20 transmits the positionalinformation of the floor surface and one wall surface, the spacecoupling unit 53 determines the coordinate transformation parameters forparallelly moving and/or rotating each local coordinate system such thatthe respective wall surfaces coincide with each other in addition to thefloor surfaces. Depending on the local coordinate system used by eachclient device 20, the coordinate transformation parameters may include aparameter for performing transformation such as mirror image inversion.

Alternatively, in a case where each of the client devices 20 transmitsinformation of three reference surfaces (i.e., one horizontal referencesurface and two vertical reference surfaces) that surround one cornerand cross each other, the space coupling unit 53 may determine thecoordinate transformation parameters for each coordinate system suchthat all the three reference surfaces respectively coincide with eachother. Note that, since sizes or shapes of the rooms where therespective users are present are generally different from each other,all the reference surfaces identified by the client devices 20 can notnecessarily coincide with each other. Therefore, the space coupling unit53 causes at least the horizontal reference surfaces to coincide witheach other and determines the coordinate transformation parameters foreach local coordinate system to further cause one or two verticalreference surfaces to coincide with each other if possible.

For example, in the example mentioned above, it is assumed thatpositional information of an adjacent wall surface P3 a adjacent to thewall surface P2 a in addition to the floor surface P1 a and the wallsurface P2 a is acquired from the client device 20 a, and positionalinformation of an adjacent wall surface P3 b adjacent to the wallsurface P2 b in addition to the floor surface P1 b and the wall surfaceP2 b is acquired from the client device 20 b. In this case, the spacecoupling unit 53 determines the coordinate transformation parameters toat least cause the floor surface P1 a and the floor surface P1 b tocoincide with each other and cause the wall surface P2 a and the wallsurface P2 b to coincide with each other. Furthermore, in a case wherean angle formed by the wall surface P2 a and the adjacent wall surfaceP3 a in the first space and an angle formed by the wall surface P2 b andthe adjacent wall surface P3 b in the second space coincide with eachother, the coordinate transformation parameters are determined to alsocause the adjacent wall surface P3 a and the adjacent wall surface P3 bto coincide with each other. In contrast, in a case where the anglesformed by the two sets of wall surfaces do not coincide with each other,since it is difficult to achieve coordinate transformation causing allthe three reference surfaces to respectively coincide with each other,the space coupling unit 53 determines the coordinate transformationparameters to only cause the floor surface P1 a and the floor surface P1b coincide with each other and cause the wall surface P2 a and the wallsurface P2 b to coincide with each other.

The space coupling unit 53 transmits the coordinate transformationparameters for each local coordinate system determined as describedabove to the corresponding client device 20. The coordinate transformingunit 54 of each client device 20 performs the coordinate transformationon the position coordinates of the object acquired by the positionalinformation acquiring unit 51 by use of the coordinate transformationparameters received from the space coupling unit 53 of the server device10. This allows the coordinate transforming unit 54 to transform theposition coordinates of each of the plurality of unit portions in thelocal coordinate system identified in the actual space into the positioncoordinates in the virtual space in the world coordinate system.

Note that the coordinate transforming unit 54 does not necessarilytransform position coordinates of all the unit portions acquired by thepositional information acquiring unit 51. For example, the coordinatetransforming unit 54 may omit transformation of position coordinates ofunit portions configuring the reference surface. Alternatively, thecoordinate transforming unit 54 may identify unit portions configuringthe body of the user according to a given algorithm to transform theposition coordinates of the unit portions. Information of thetransformed position coordinates is transmitted to the server device 10.

The space updating unit 55 functions as a virtual space builder andupdates details of the virtual space by use of the information of theposition coordinates of the object received from each client device 20.Specifically, the space updating unit 55 disposes an avatar Aarepresenting the user A in the virtual space by use of the positioncoordinates, which are received from the client device 20 a, of eachunit portion configuring the user A. Further, the space updating unit 55disposes an avatar Ab representing the user B in the virtual space byuse of the position coordinates, which are received from the clientdevice 20 b, of each unit portion configuring the user B. Herein, theposition coordinates received from each client device 20 are theposition coordinates in the world coordinate system after beingtransformed according to the coordinate transformation parametersdetermined by the space coupling unit 53. Therefore, a position and aposture of the avatar to be disposed in the virtual space can bedetermined by use of the received position coordinates without change.

The avatar of each user thus disposed is updated while reflecting theposture and the movement of the user in the actual space. Herein, thecoordinates of the position of each user is transformed by use of thecoordinate transformation parameters determined by the space couplingunit 53. This allows the avatars of the plurality of users to share ahorizontal reference surface having an identical height. For example, ina case where the floor surface of the room where each user is present isidentified as the horizontal reference surface, the avatars of the usersare disposed in the virtual space so as to stand on the floor surfacehaving the identical height.

Alternatively, in a case where an upper surface of a table is identifiedas the horizontal reference surface, the space coupling unit 53determines the coordinate transformation parameters such that thehorizontal reference surfaces of the local coordinate systems coincidewith each other as described above. In this case, the space updatingunit 55 disposes a table object such that its upper surface coincideswith a position, in the virtual space, corresponding to the horizontalreference surface. This allows the plurality of users to share the tablehaving the identical height in the virtual space. In other words, whenthe user A touches the upper surface of the table present in his or herroom with his or her hand, the avatar of the user A also touches theupper surface of the table object disposed in the virtual space with itshand. Similarly, when the user B touches the upper surface of the tablepresent in his or her room with his or her hand, the avatar of the userB can touch the upper surface of the identical table object disposed inthe virtual space with its hand. Herein, even when heights of the tablesactually present in the rooms where the respective users are present aredifferent from each other, the coordinate transformation is performedsuch that the upper surfaces of the tables coincide with each other.This allows the identical table object to be shared in the virtualspace. Therefore, each of two users can touch the table object or avirtual object disposed on the table object in the virtual space, forexample, as if the table in his or her room is shared with the otheruser who is present in a remote place.

In a case where the space coupling unit 53 determines the coordinatetransformation parameters so as to share one or more vertical referencesurfaces, the space updating unit 55 disposes wall surface objects atpositions, in the virtual space, corresponding to the vertical referencesurfaces. This allows each of the plurality of users to share one ormore wall surfaces in the virtual space with another user who is presentin a remote location similarly to the floor surface or the upper surfaceof the table.

FIG. 4 is a view illustrating a state of the virtual space updated bythe space updating unit 55 in this manner. In the example in FIG. 4, theavatar Aa representing the user A and the avatar Ab representing theuser B are disposed in one virtual space. Furthermore, the coordinatetransformation parameters are determined such that the floor surface P1a in a space where the user A is actually present and the floor surfaceP1 b in a space where the user B is actually present coincide with eachother, and such that the wall surface P2 a in the space where the user Ais present and the wall surface P2 b in the space where the user B ispresent coincide with each other. With this configuration, in thevirtual space, a floor surface P1 v corresponds to both the floorsurface P1 a and the floor surface P1 b, and a wall surface P2 vcorresponds to both the wall surface P2 a and the wall surface P2 b.Therefore, when the user A touches the wall surface P2 a with his or herhand in the actual space, the avatar Aa touches the wall surface P2 vwith its hand, and when the user B touches the wall surface P2 b withhis or her hand in the actual space, the avatar Ab touches the wallsurface P2 v with its hand. With this configuration, when the user A andthe user B perform an operation in the actual space such as tracing thewall surface present nearby with his or her hand, for example, the userA and the user B can depict characters or pictures on the wall surfaceP2 v in the virtual space.

Note that, herein, it is assumed that the avatars representing theplurality of users are disposed on the same side with respect to thevertical reference surface in the virtual space. However, the pluralityof avatars may be disposed on opposite sides with respect to onevertical reference surface. For example, in a case where two personsplay tennis, they face each other over a net, and it is not assumed thateach person enters a court on the opposite side. In a case where such asituation is reproduced, the space coupling unit 53 determines thecoordinate transformation parameters for each client device 20 such thatthe vertical reference surfaces identified by the two client devices 20reversely coincide with each other in the virtual space (i.e., such thatsurfaces of the vertical reference surfaces directed to correspondingusers are opposite to each other). By disposing the avatar of each userin the virtual space according to the position coordinates in the worldcoordinate system transformed with such coordinate transformationparameters, the user can stand facing the wall in the room where theuser is present and face the avatar of the other user present on theopposite side of the wall in the virtual space.

The space updating unit 55 may also determine a position, a shape, and asize of another virtual object disposed in the virtual space by use ofthe information of the position coordinates of the unit portionsreceived from each client device 20, in addition to the avatar of eachuser. For example, in a case where a cabinet is placed in the room wherethe user A is present, the space updating unit 55 disposes a virtualobject having a shape similar to that of the cabinet also in the virtualspace. Such control can prevent each user from entering a location wherethe object is present in the actual space.

In a case where a wall surface other than the surface selected, by thespace coupling unit 53, as the reference surface to be shared ispresent, the space updating unit 55 may dispose a wall surface objectcorresponding to the wall surface in the virtual space. Specifically,for example, in a case where one wall surface is identified as thevertical reference surface, a position of a wall surface facing thiswall surface in the actual space varies depending on the size of theroom where each user is present. In this case, the space updating unit55 disposes the wall surface object at a position, in the virtual space,corresponding to a wall surface closest to the reference surface amongwall surfaces facing the reference surface in the room where each useris present. This can restrict a movable range recognized by each user inthe virtual space according to the size of a narrower room of the roomswhere respective users are actually present. With this configuration, itcan be produced that no one can enter a space where any one of theplurality of users cannot enter.

The space updating unit 55 delivers information of each virtual objectdisposed in the virtual space based on information of an object, whichis received from each client device 20, to each client device 20. Thespatial image depicting unit 56 of each client device 20 depicts aspatial image indicating a state in the virtual space by use of theinformation received from the space updating unit 55 and causes thedisplay device 55 to display the spatial image. At this time, the spaceupdating unit 55 identifies a position of the head of the user who viewsthe displayed spatial image and disposes a virtual camera at a positionwhere the identified position is transformed into the world coordinatesystem. The spatial image depicting unit 56 depicts a spatial imageindicating a state of the inside of the virtual space viewed from thedisposed virtual camera. This allows each user to view the state of theinside of the virtual space shared with another user who is present in aremote location. This spatial image may include an avatar representinganother user.

Hereinafter, an example of a flow of processing performed by theinformation processing system 1 will be described using a flow diagramin FIG. 5.

First, as initial processing, the positional information acquiring unit51 of each client device 20 acquires positional information of an objectaround the user (S1). The positional information acquiring unit 51 thenidentifies a position of a reference surface in each local coordinatesystem by use of the acquired positional information (S2). Thepositional information of the identified reference surface istransmitted to the server device 10 (S3).

The space coupling unit 53 of the server device 10 determines thecoordinate transformation parameters for transforming the localcoordinate system of each client device 20 into the world coordinatesystem by use of the positional information of the reference surface,which is transmitted from each client device 20 in S3 (S4). As describedabove, the space coupling unit 53 determines the coordinatetransformation parameters used by each client device 20 such that thepositions of the reference surfaces transmitted from the client devices20 coincide with each other in the virtual space. The space couplingunit 53 transmits the coordinate transformation parameters determined inS4 to each client device 20 (S5), thereby terminating the initialprocessing.

Hereinafter, the information processing system 1 repeats acquisition ofthe positional information of each user and update of the virtual spaceaccording to the acquired positional information. Specifically, thepositional information acquiring unit 51 of each client device 20acquires positional information of current objects including at leastthe body of the user (S6). The coordinate transforming unit 54transforms the positional information acquired in S6 into the positioncoordinates in the world coordinate system by use of the coordinatetransformation parameters received in S5 (S7) and transmits informationof the transformed position coordinates to the server device 10 (S8).

The space updating unit 55 of the server device 10 updates informationof a virtual object to be disposed in the virtual space by use of theposition coordinates received from each client device 20 in S8 (S9).Specifically, the space updating unit 55 updates at least a position anda posture of an avatar corresponding to each user by use of thepositional information of the user, which is received in S8. Thereafter,the space updating unit 55 transmits positional information of eachvirtual object in the updated virtual space to each client device 20(S10).

The spatial image depicting unit 56 of each client device 20 depicts aspatial image indicating a state of the inside of the virtual space byuse of the positional information of the virtual object received in S10and causes the display device 25 to display the spatial image (S11). Byrepeating the processing from S6 to S11, a state of the virtual spacewhere a plurality of avatars each of which reflects movement of the bodyof the corresponding user are present can be presented to each user.

According to the above-described exemplary embodiment of the presentdisclosure, the coordinate transformation is performed according to thecoordinate transformation parameters determined such that the referencesurfaces identified in the places where the users are individuallypresent coincide with each other. Therefore, each user can share thereference surface in the place where he or she is present with anotheruser present in a remote place in one virtual space. With thisconfiguration, each user can share the identical virtual space withanother user present in the remote place and, at the same time, achieveinteraction in which each user and another user touch and operate anidentical object in the virtual space. Such a technology can be used forroom-scaled communication by a plurality of persons, by use of desks orwalls, such as a remote meeting system.

Note that exemplary embodiments of the present disclosure are notlimited to the exemplary embodiment described above. For example, in theabove description, the images captured by the stereo camera 26 are usedto identify the positions of the user and the reference surface presentaround the user. However, the exemplary embodiments are not limited tothis configuration, and the client device 20 may use, for example, atime-of-flight (TOF) type sensor to detect positions of objects presentin the actual space. Alternatively, movement of the user or thepositions of the objects present around the user may be identified byuse of, for example, the simultaneous localization and mapping (SLAM)technology according to movement of an image captured by a cameramounted on the display device 25. Alternatively, the positions of theobjects may be detected using a technology such as Patterned Stereo orStructured Light. Furthermore, the client device 20 may identify theposition of the user and the position of the reference surface bycombining detection results of a plurality of sensor devices disposed inthe actual space.

In the above description, the coordinate transformation from the localcoordinate system to the world coordinate system is performed for eachunit portion identified using the image captured by the stereo camera26. However, the coordinate transformation may be performed with anotherunit. For example, the client device 20 may estimate a skeleton model ofthe body of the user using the image captured by the stereo camera 26and may perform the coordinate transformation on position coordinates offeature points configuring the estimated skeleton model.

Alternatively, at least part of processing to be performed by the clientdevice 20 in the above description may be performed by the server device10. As an example, the coordinate transformation from the positioncoordinates in the local coordinate system to the position coordinatesin the world coordinate system may be performed by the server device 10.In this case, the server device 10 need not transmit the coordinatetransformation parameters to each client device 20. In contrast, atleast part of processing to be performed by the server device 10 in theabove description may be performed by the client device 20.Alternatively, any one of the plurality of client devices 20 may alsofunction as the server device 10 in the above description.

It should be understood by those skilled in the art that variousmodifications, combinations, sub-combinations and alterations may occurdepending on design requirements and other factors insofar as they arewithin the scope of the appended claims or the equivalents thereof.

What is claimed is:
 1. An information processing device comprising: anacquiring unit that acquires positional information of: a firsthorizontal surface, a first vertical surface, and a first adjacentvertical surface adjacent to the first vertical surface present in afirst space around a first user; a second horizontal surface, a secondvertical surface, and a second adjacent vertical surface adjacent to thesecond vertical present in a second space around a second user; and atransformation parameter determining unit that determines a coordinatetransformation parameter for transforming position coordinates of thefirst space and the second space into position coordinates in a virtualspace such that: wherein, in a case where an angle formed by the firstadjacent vertical surface and the first vertical surface and an angleformed by the second adjacent vertical surface and the second verticalsurface coincide with each other, the transformation parameterdetermining unit determines the coordinate transformation parameter suchthat: the first horizontal surface and the second horizontal surfacecoincide with each other, the first vertical surface and the secondvertical surface coincide with each other, and the first adjacentvertical surface and the second adjacent vertical surface coincide witheach other, and wherein, in a case where the angle formed by the firstadjacent vertical surface and the first vertical surface and the angleformed by the second adjacent vertical surface and the second verticalsurface do not coincide with each other, the transformation parameterdetermining unit determines the coordinate transformation parameter onlysuch that: the first horizontal surface and the second horizontalsurface coincide with each other, and the first vertical surface and thesecond vertical surface coincide with each other, wherein a position ofan object present in the first space and a position of another objectpresent in the second space are transformed into positions in thevirtual space according to the determined coordinate transformationparameter.
 2. The information processing device according to claim 1,wherein the first horizontal surface and the second horizontal surfacesare flat surfaces.
 3. The information processing device according toclaim 1, wherein the acquiring unit acquires positional information of aplurality of flat surfaces present in the first space and positionalinformation of a plurality of flat surfaces present in the second space,and the transformation parameter determining unit determines thecoordinate transformation parameter such that the plurality of flatsurfaces present in the first space and the plurality of flat surfacespresent in the second space respectively coincide with each other. 4.The information processing device according to claim 1, wherein thetransformation parameter determining unit determines the coordinatetransformation parameter such that a surface on a first user side of thefirst vertical surface and a surface on a second user side of the secondvertical surface face opposite to each other.
 5. The informationprocessing device according to claim 1, further comprising: a virtualspace building unit that builds a virtual space including a referencesurface corresponding to the first horizontal surface present in thefirst space and the second horizontal surface present in the secondspace, wherein the first horizontal surface and the second horizontalsurface have a different size, wherein the virtual space building unitrestricts a movable range of a user in the virtual space to a rangecorresponding to a narrower space than the first horizontal surface orthe second horizontal surface.
 6. An information processing systemcomprising: a plurality of client devices; and a server device, whereinthe server device includes an acquiring unit that acquires, from each ofthe plurality of client devices, positional information of: a horizontalsurface present in a space around a user of the client device, avertical surface present in the space around the user of the clientdevice, and an adjacent vertical surface in the space around the user ofthe client device; and a transformation parameter determining unit thatdetermines a coordinate transformation parameter for transforming aposition coordinate of a space around a user who uses corresponding oneof the plurality of client devices into a position coordinate in avirtual space such that: for each of the plurality of client deviceswhere an angle formed by the vertical surface and the adjacent verticalsurface coincide with each other, the coordinate transformationparameter is determined such that the horizontal surfaces present in thespaces coincide, the vertical surfaces present in the spaces coincide,and the adjacent vertical surfaces present in the spaces coincide, andfor each of the plurality of client devices where an angle formed by thevertical surface and the adjacent vertical surface do not coincide witheach other, the coordinate transformation parameter is determined onlysuch that the horizontal surfaces present in the spaces coincide, andthe vertical surfaces present in the spaces coincide.
 7. An informationprocessing method comprising: acquiring positional information of: afirst horizontal surface, a first vertical surface, and a first adjacentvertical surface adjacent to the first vertical surface present in afirst space around a first user a second horizontal surface, a secondvertical surface, and a second adjacent vertical surface adjacent to thesecond vertical present in a second space around a second user; anddetermining a coordinate transformation parameter for transformingposition coordinates of the first space and the second space intoposition coordinates in a virtual space such that: wherein, in a casewhere an angle formed by the first adjacent vertical surface and thefirst vertical surface and an angle formed by the second adjacentvertical surface and the second vertical surface coincide with eachother, the coordinate transformation parameter is determined such that:the first horizontal surface and the second horizontal surface coincidewith each other, the first vertical surface and the second verticalsurface coincide with each other, and the first adjacent verticalsurface and the second adjacent vertical surface coincide with eachother, and wherein, in a case where the angle formed by the firstadjacent vertical surface and the first vertical surface and the angleformed by the second adjacent vertical surface and the second verticalsurface do not coincide with each other, the coordinate transformationparameter is determined only such that: the first horizontal surface andthe second horizontal surface coincide with each other, and the firstvertical surface and the second vertical surface coincide with eachother wherein a position of an object present in the first space and aposition of another object present in the second space are transformedinto positions in the virtual space according to the determinedcoordinate transformation parameter.
 8. An information processing devicecomprising: an acquiring unit that acquires positional information of: afirst vertical surface and a first adjacent vertical surface facing thefirst vertical surface present in a first space around a first user, anda second vertical surface and a second adjacent vertical surface facingthe second vertical present in a second space around a second user; anda transformation parameter determining unit that determines a coordinatetransformation parameter for transforming position coordinates of thefirst space and the second space into position coordinates in a virtualspace such that: the first vertical surface and the second verticalsurface coincide with each other at a first wall surface in the virtualspace, and wherein, in a case that a distance d1 between the firstvertical surface and the first adjacent vertical surface is greater thana distance d2 between the second vertical surface and the secondadjacent vertical surface, a second wall surface is set the distance d2from and facing the first wall surface in the virtual space, wherein, ina case that the distance d2 is greater than the distance d1, the secondwall surface is set the distance d1 from and facing the first wallsurface in the virtual space.
 9. The information processing device ofclaim 8, wherein the acquiring unit further acquires positionalinformation of a first horizontal surface in the first space and asecond horizontal surface in the second space, and wherein thetransformation parameter determining unit determined the coordinatetransformation parameter such that the first horizontal surfacecoincides with the second horizontal surface.